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Standard definition of the stochastic Risk- Sensitive Linear-Quadratic (RS-LQ) control depends 
on the risk parameter, which is normally left to be set exogenously. We reconsider the classical 
approach and suggest two alternatives resolving the spurious freedom naturally. One approach 
consists in seeking for the minimum of the tail of the Probability Distribution Function (PDF) of 
the cost functional at some large fixed value. Another option suggests to minimize the expectation 
value of the cost functional under constraint on the value of the PDF tail. Under assumption of the 
resulting control stability, both problems are reduced to static optimizations over stationary control 
matrix. The solutions are illustrated on the examples of scalar and Id chain (string) systems. Large 
Deviation self-similar asymptotic of the cost functional PDF is analyzed. 

I. INTRODUCTION 

Stochastic differential equations are used both in control [H-0] and statistical physics [8l4l^| to state the problems. 
The two fields also use similar mathematical methods to analyze these equations. However, and in spite of the 
commonalities, there were relatively few overlaps between the disciplines in the past, even though the communications 
between two communities improved in the recent years. Some new areas in control, for example stochastic path 
integral control [L3l4l6j . have emerged influenced by analogies, intuition and advances in statisical/theoretical physics. 
Vice versa, many practical experimental problems in physics, chemistry and biology dealing with relatively small 
systems (polymers, membranes, etc), which are driven and experience significant thermal fluctuations, can now be 
analyzed and manipulatcd/controled with accuracy and quality unheard of in the past, see for example fl7l fl8j . 
Besides, approaches from both control theory and statistical physics started to be applied to large natural and 
engineered networks, like chemical, bio-chemical and queuing networks pJU22]. Dynamics over these networks is 
described by stochastic differential equations, the networks have enough of control knobs, and they function under 
significant fluctuations which need to be controlled to prevent rare but potentially devastating failures. Related 
setting of stochastic optimization, i.e. optimization posed under uncertainty, has also came recently in the spot 
light of statistical physics inspired algorithms and approaches [23j . Convergence of these are related ideas motivated 
the manuscript, where we discuss analysis and control of rare events in the simplest possible, but practically rather 
widespread universal and general, linear setting. We realize that the general topic of linear control is well studied and 
many (if not all) possible questions, e.g. related to proper way of accounting for risk (rare events), were discussed in 
the field in the past. In spite of that, we still hope that this manuscript may also be useful not only to physicists, 
who may wish to explore new and largely unusual (in physics) formulations, but also to control theorists. 

Consider first order (in time derivatives) stochastic linear dynamics of a vector x = (xi\i = 1, • • • , N) over time 
interval t' £ [t; T] 

^-x = Ax + Bu + f(f), (LI) 
at' 

where A and B are constant matrices; u(t') is the control vector applied at the moment of time t'\ and {£} = (£(t')|f' € 
[t;T]) is the zero mean, short-correlated noise with covariance V 

&(O> = 0, m%(t")) = 6(t'-t")V ij , i,j = l,-..,N (1.2) 

where one utilizes "statistical physics" notations for the expectation value (average) over noise, (■••). Here in Eq. (|1.2[) 
and below the averages are over multiple possible realizations of the noise, each generating a new trajectory of the 
system, {x} = (x(t')\t' £ [t;T]), under given control {u} = (u(t')\t' <G The Eq. (|1.1[) is causal, thus assuming 

retarded (Stratonovich) regularization of the noise on the right-hand-side of the discreet version of Eq. (|1.1[) . The 
physical meaning of the vectors and matrices in Eq. (|1.1[) is as follows. A is the matrix explaining stretching, shearing 
and rotation of the system trajectory in the A^-dimensional space if the control and external noise would not be 
applied. Matrix B describes possible limitations on the degrees of freedom in the system one can control. To simplify 
notations we consider signal, control and noise vectors having the same dimension, N, where thus B is quadratic. 
The setting of Eqs. (|l.lll.2[) is classic one in the control theory. It describes the so-called Linear-Quadratic (LQ) 
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stochastic control problem, which was introduced in 0, 0, 0, Q and became foundational for the control theory as a 
field, see e.g. [13, HE] and references therein. In the classical formulation one seeks to solve the following optimization, 
t€[0;T\: 



mm 
M \ 



^J(t;T;{u},{x})J, (1.3) 
J(t; T; {u}, {x}) = l -x*(T)Fx{T) + i ^ dt' (x* (t')Qx(t') + u* (t')Ru(t')) , 



where Q, R and i* 1 are pre-defined stationary (time independent) symmetric positive matrices and one uses the super- 
script asterisk, *, to mark transposition. J(t; T; {u}, {x}) (later on, and when it is not confusing, we will use the 
shortcut notation J) is a scalar quadratic cost functional of the state vector {x} = (x(t')\t' € [i;T]), and the control 
vector, {u} = (u(t')\t' £ [t',T]) evaluated for all intermediate times if from the [0; T] interval. Here in Eqs. (|1.3[) 
(and everywhere below in the manuscript) the average over noise {£} includes conditioning to Eq. (|1.3[) . i.e. {x} is 
dependent on realization of the noise, {£}, and on the control, {it}, according to Eq. (|1.1[) . It is assumed that the 
stochastic LQ control is evaluated off-line, i.e. the optimal solution u„ (t; x{t)) of Eq. (|1.3p is computed and saved prior 
to executing actual experiment for any initial condition x(i) at any i. Then in the course of the actual experiment 
(execution of the dynamics) x(t) is measured at any time i and respective u*(t;x(t)) is applied. (When observation 
of x(t) is partial and noisy one needs to generalize the stochastic LQ control, for example considering the stochastic 
Linear Quadratic Gaussian (LQG) control, see e.g. [H| for details.) We also assume (and the details will be clarified 
below) that the optimal control succeeds, i.e. the systems stabilizes and J does not grow with T faster than linearly. 

An unfortunate caveat of the LQ setting (|1.3p is in the lack of fluctuations control: even though the LQ solution is 
optimal in terms of minimizing the expectation value of the cost functional it may generate very significant fluctuations 
when it comes to analysis of the J 3> ( J) tail of the Probability Distribution Function, V(J), of the cumulative cost 
f7(T;x(0)) = J(0; T; {it*}, {x}). Stochastic Risk Sensitive LQ (RS-LQ) scheme j26l - l28j was introduced to improve 
control of the abnormal fluctuations of J. RS-LQ constitutes the following generalization of the LQ scheme (jTT 



max(exp(-0J)}, (1.4) 
M 

where 6 is a pre-defined parameter. Intuitively one relates the case of positive 8 to a risk-avert optimum. It is assumed 
within the standard RS-LQ scheme that 8 is fine tuned by some additional considerations. Note that, as shown in 
[29| . the RS-LQ control is also ultimately related to the so-called Hoc-norm robust control. (See also [3(| for further 
discussion of the relation.) 

In this paper we analyze two natural modifications of the stochastic RS-LQ control. The two schemes can both 
be interpreted in terms of the RS-LQ approach supplemented by an additional optimization over 8. Our first, Tail- 
Optimum (TO), scheme consists in the following modification of the LQ (|1.3|) and RS-LQ (|1.4p ones 

mmV(J = j-(T-t)\{u}). (1.5) 

M 

In words, the TO-LQ control minimizes (at any time t and given the current observation x(t)) the probability of the 
current value of the cost functional J(t;T; {u}, {x}) evaluated at a predefined value, j ■ (T — t), where thus j is the 
only external parameter left in the formulation. Another strategy, which we call Chance- Constrained LQ (CC-LQ), in 
reference to similar formulations in optimization theory [3l| - [33j , consists in minimizing the mean of the cost functional 
under condition that the tail probability evaluated at j ■ (T — t) does not exceed the prescribed threshold value e(t; T) 

mm (J) (1.6) 

{«} 

s.t. V(J = j-(T-t)\{u})<s(t;T). 
Main objectives, and consequently results of this study, are 

To extend the asymptotic, T — > oo, approach, developed in the past for LQ \1.3\) and RS-LQ Jj.^[ ) optimal controls to 
the new TO-LQ \1.5\l and CC-LQ \1.6\ ) optimal settings. At T — > oo the optimal control takes the following universal 
linear in x form 

u*(i; x) = —Kx, (1.7) 



where K is i-independent but model dependent matrix. The condition of the system stability, intuitively translating 
into the expectation that J grows not faster than linearly with T, naturally requires that all the eigenvalues of the 
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stability matrix, /i = BK — A, have positive real part. The linearity of the optimal control p. 71) in x is a direct 
consequence of the linearity of the initial dynamical system. Time-indepedence and initial condition-independence of 
the optimal control (|1.7|) are asymptotic: they are achieved at T > t,, where t* can be estimated as the inverse of 
the absolute value of the (BK — A)'s eignevalue with the smallest real part. System of algebraic equations defining 
K implicitly for the TO-LQ and CC-LQ cases are presented and then juxtaposed against the previously analyzed 
cases of the LS and RS-LQ controls. (See Eqs. (|4.1|4.3|4.5jl .) Finding optimal control is reduced to optimization over 
time- independent, K. The resulting dependencies are homogeneous in time, with t and T always enter in the T — t 
combination. (This also simplifies the analysis allowing to set t = 0.) 

• To analyze statistics of the optimal cost functional, J , in the Large Deviation (LD) regime, i.e. at large but finite 
T . We show that in the stable regime the PDF of J attains the following universal LD form 

log W) ~ -TS(J/T), (1.8) 

where the LD function, S(j), is a convex function of its argument found implicitly (in a closed algebraic form, which 
may or may not yield an efficient algorithm) for the four cases (of LS, RS-LQ, TO-LQ and CC-LQ controls) considered. 
The LD function shows a universal, S(j) — > aj, tail at large (i.e. larger than typical) j, where the value of positive 
a depends on the model. This suggests, in particular, that it is natural to choose in the CC optimization (|1.6|) . 
e(t;T) = exp(— c(T — i)), for the threshold, with c been a constant. To derive compact algebraic expressions for the 
LD function we, first, analyze the generating function of J evaluated at linear u parameterized by K as in Eq. (|1.7I) . 

Z(6;K) = {exp(-6J))., (1.9) 

then express the optimization/control objective as a convolution of the integral or differential operator/kernel in 8 
(the choice will depend on the model) and 2(8; K), and finally evaluate optimization over K in the asymptotic LD 
approximation. Here in Eq. (|1.9j) the low asterisk mark * in the expectation/average (over noise and constrained to 
Eq. (jl.ip ) indicates that the control vector is taken in the form of the linear ansatz, u — > —Kx, where K is left yet 
undetermined. 

The remainder of the manuscript is organized as follows. We start discussing the deterministic case (of zero noise) in 
Section[TTl This regime is of interest for two reasons. First, in the asymptotic of zero noise the four, generally different, 
control schemes become equivalent. Besides, and as well known from the classical papers [1, HH HI| > optimal control in 
the bare LQ case (correspondent to minimization of the cost function average) is not sensitive (and thus independent 
of) the level of the noise. Section IrLTl is devoted to analysis of the generating function (|1.9p . the average value of the 
cost function and the tail of the cost function distribution restricted to yet unspecified value of K. Optimization over 
K, resulting in the known RS-LQ optimal relations and also derivation of the new optimal relations for K in the 
TO-LQ and CC-LQ cases, is discussed in Section IIVI We describe and compare asymptotic Large Deviation forms 
of the cost function PDF, V(J\ in the optimal regimes. In this and preceding Section we also discuss many times 
the illustrative "scalar" example, where x and u are scalars. An infinite system example, of a "string" formed from a 
linear Id chain, is discussed in Section [Vj We conclude and discuss related future challenges in Section IVT1 



II. DETERMINISTIC CASE AND LQ-OPTIMAL CONTROL 

We start this Section from a disclaimer: all results reported here are classical, described in 0, l24l - [29| and latter 
papers and books, see e.g. [3343^ |. We present it here only for making the whole story of the manuscript self- 
explanatory and coherent. 

When the noise is ignored, Eq. (|1.1[) should be considered as a deterministic constraint, reducing any of the optimal 
control schemes (|1.3I1.4I1.5I1.6|) to a simple variation of the cost functional (|1.4p over u. Using the standard variational 
technique with a time dependent Lagrangian multiplier for the constraint, and then excluding the multiplier one derives 
the equation 

4iu* +u*RB- 1 ABR- 1 =x*QBR' 1 , (2.1) 
at' 

which should be supplied by the boundary condition (also following from the variation), u*(T) + x*(T)FBR~ 1 = 0. 
(Let us remind that we choose the notations where the dimensionality of u coincides with the dimensionality of x. 
We also assume that inverses of all the matrices involved in the formulation are well defined. This assumption is not 
critical and is made here only to simplify the notations. In the general case when some of the matrices, in particular R, 
are not full rank, one can generalize the formulas properly, using a proper notion of the pseudo-inverse.) Substituting, 
u = —R~ 1 B*Hx, in Eq. (|2.1j) one arrives at the following equation for II 



— n + IL4 + A*n ~ UBR^B*!! + Q = 0. 
dt' ^ 



(2.2) 
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with the boundary condition II(T) = F. Eq. (|2.2[) . solved backwards in time, results in IT(i) and then, u*(t;x) = 
-R- x B*n{t)x = -Kx. 

To gain a qualitative understanding of the backwards in time dynamics of IT, let us briefly discuss the simplest 
possible case with all the matrixes entering Eq. (|2.2p replaced by scalars, then yielding the following analytic solution 
for the optimal K 

A jA^Qf- ( tanh U A* + ^(t -T ))) * 
K= ^ \ U— t (2.3) 



where T and ±1 are chosen to satisfy the boundary condition, K(T) = BF/R. When T r = 1/^/A 2 + QB 2 /R, 
the backwards in time dynamics saturates (after a short ~ t transient) to a _F-independent constant, resulting from 
replacing tanh in Eq. (|2.3|) by —1. Therefore, in the stationary regime, T — ¥ oo the optimal control is with the constant 
in time, frozen K . One also finds that the optimal control in the one dimensional deterministic case is always stable, 
IX = KB - A > 0. 

Returning to the general (finite vector) case one concludes that when T is sufficiently large the optimal control is 
of the form described by Eq. (|1.7[) , i.e. it is linear in x and asymptotically time independent, with K = i? _1 B*rio 
where Ho solves Eq. (|2.2j) with the first term replaced by zero. It is well known in the control theory that (under 
some standard common sense assumptions on B and R matrices) stable solution of the system of the algebraic Riccati 
equations is unique and moreover it can be found efficiently. (See e.g. Chapter 12 of Sec[I3] and references therein.) 

Let us now discuss the bare LQ control, now in the presence of the noise. Since Eq. (|1.1[) is linear, one can naturally 
split the full solution into a sum, x = x\ + X2, where x\ satisfies Eq. without noise and it is equivalent to the 
noise- less solution, just discussed in this Section. Then, the second term satisfies, dx2/dt' = Ax-i + £, with X2(t) = 0. 
However, since the noise is zero mean, (£) = 0, X2 is zero mean too, i.e. (X2) — 0. Next, let us analyze the split of 
term in (J), which is the optimization objective of the LQ scheme. Since, x\ and X2 are independent (by construction) 
and because X2 is zero mean, (J), splits into two terms, (J\) + (J2), each dependent on x\ and X2 vectors only. (Ji) 
is simply equivalent to J analyzed above in the deterministic case, while [J 2) is it- independent, thus not contributing 
the optimization at all. To summarize, the LQ optimal control is not sensitive to the noise and it is thus equivalent 
to the deterministic (noiseless) case described above. 



III. GENERATING FUNCTION 



Consider the Generating Function (GF), Z(6;K), defined by Eq. (|1.9p . Z(9;K) is of an obvious relevance to the 
RS-LQ scheme, but it is also useful for analysis of other schemes as well, because of the following (Laplace transform) 
relation to the PDF of J: 

OO 

Z(9;K) = JdJ exp(-ej)V*(J), (3.1) 


where (as before) the asterisk in the sub-script indicates that the PDF was evaluated at u = Kx, with K being yet 
undefined constant matrix. The inverse of Eq. (|3.1[) is 

c+ioo 

W)= / ^-cxp(ej)Z(6;K), (3.2) 

J 27TI 

c—ioo 

where it is assumed that the integration contour, considered in the complex plain of 9, goes on the right from all the 
singularities (poles and cuts) of Z(6; K). In the path integral representation GF gets the following form 

Z(6;K)~ j VxVp expU dt \-^x*Qx + p*(d t x + fix) + ^P*Vpj \ (3.3) 

= + K*RK, fi = BK - A, (3.4) 

where p is an auxiliary vector variable (momentum). Here and everywhere below we assume that, even if the dynamics 
was not stable before application of the control, control stabilizes it. Formally, this means that n, defined by Eq. (13.41) . 
has no eigenvalues with negative real values. The "boundary" (_F-dependcnt term) in Eq. (|3.3p was ignored, assuming 
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that (like in the one-dimensional LQ case discussed above) it may only influence how the optimum is approached 
(backwards in time) but remains inessential for describing asymptotic behavior of the optimal control. This path 
integral is (most conveniently) evaluated by changing to the Fourier (frequency) domain, expressing pair correlation 
function as the frequency integral, and then relating it to the derivative of the log-GF over 9, 



+00 

duj 



2 

dlogZ(9:K) T 



— OO 



-(tJ i V- x +n*V-' l n + 0Q\ \ (3.5) 

7T V / ij 

x*Qx). (3.6) 



89 2 

Here in Eqs. (|3.5I3.6[) the averaging is over the path integral measure described by Eq. (|3.3p . Further, evaluating the 
integral over 9, fixing normalization, Z(0; K) = 1, and using the standard formula of matrix calculus, d/d6 log det(# • 
1 + D) = tr((0 • 1 + -D) -1 ), where 1 stands for the unit matrix, one arrives at the following expression 



= -2 / 2^ det(^- 1+/ ^-V) 



(3.7) 



which is asymptotically exact at T — > 00. Moreover, one can show that for any (spatially) finite system next order 
corrections to the rhs of Eq. (|3.7[) are 0(1). Note that this representatio n (I3.7[) of the log-GF, as an integral over 
frequency of a log-det, is similar to the relation discussed in Section 3 of [29[ in the context of linking the RS-LQG 
control to the maximum entropy formulation of the Hoo control. The log-det has also appeared in [38| where statistics 
of currents were analyzed in general non-equilibrium (off-detailed-balance) linear system. 

To gain intuition let us first analyze Eq. (|3.7p in the simple scalar case where the integral on the rhs can be evaluated 
analytically 



log(Z(0; K)) = I (V - y/fjt + OQV) . (3.8) 

Substituting this expression into Eq. Q3.2p and estimating the integral over 9 in a saddle-point approximation (justified 
when T is large) one arrives at the LD expression (|1.8[) where 

V{Q + RK 2 ) (BK - A) 2 j BK-A 
S * {J) = 16j + V(Q + RK 2 ) —2—- (3 - 9) 

The LD function is obviously convex and it is defined only for positive j. (The asterisk marks, as before, that 
the average and the probability are computed conditioned to yet unspecified K.) iS*(j) achieves its minimum at, 
(j)* = — T^ 1 dlogZ/d9\g =0 = VQ/(4fi), and shows linear asymptotic, S*(j) ps iy?/{VQ), at j 3> (j). Note, that 
the aforementioned asymptotic is associated with the cut-singularity in the complex 9 plane of the GF expression 
(|3.8p . Indeed, substituting Eq. (|3.8p into Eq. (|3.2p and shifting the integration contour to the left, thus forcing it 
to surround anti-clockwise the ] — oo;9* = — ^l 2 /(VQ)] cut, and then estimating the integral by a small part of the 
contour surrounding vicinity of the cut tip at 9*, we arrive at the aforementioned j 3> (j) asymptotic, S(j) ~ —j9*. 

Returning back to analysis of the general formulas (|3.7l3.2p . one observes that even though to reconstructing S*(j) 
in its full integrity explicitly as a function of K does not look feasible, we can still, motivated by the scalar case 
analysis, make some useful general statements about both the average, (j)*, and the j 3> (j)* asymptotic of 5*(j). 
We will start from the latter problem. 

For analysis of the tail the key object of interest is the det in Eq. Q3.7P considered at zero frequency, u> = 0. 
Specifically, one aims to find the zero of the determinant with the largest real value: 

9 * = maxRe (#) dot (^ y - v+e g) =0 ■ (3-10) 

Indeed, any zero (there might be many of these in the general matrix case) marks the tip of the respective cut 
singularity of Z(9;K) in the complex (9-plane. Then, the tail, j ^ (j)*, asymptotic of the LD function becomes, 
5*(j) = —j9*. Note, that this linear in j estimation is valid only in the case of a finite system, when the set 
(spectrum) of zeros (defined by the condition in Eq. (|3.10p is discrete. In the case of an infinite system, when the 
spectrum of zeros becomes quasi-continuous, one needs to account for the multiple zeros, as illustrated in the "string" 
example of Section [Vj 
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To evaluate (j)* (as a function of K) in the general case one first analyzes it in the time representation. Substituting 
the u = —Kx ansatz with constant K in Eq. expressing x(t) formally as an integral over time (for a given 

realization of the noise), substituting the result into Eq. (|1.4j) . averaging over noise, and then taking the T — > oo limit 
one arrives at 

oo 

0% - \ I tr (Ve-^Qe-^) dt = ±tv (^n)|^ n+n>=(5 , (3.11) 
o 

where the latter expression is implicit (as the condition is a matrix one, thus not resolvable explicitly in general) 
function of K. It is straightforward but tedious to check (introducing matrix Lagrangian multiplier for the condition 
in Eq. p. lip and making variation over K and II) that optimization of Eq. (|3.11[) over K results in the algebraic 
Riccatti equation equivalent to Eq. (|2.2[) with the first term ignored. Note that the fact that the optimal control 
derived from the optimization of the average cost function in the stochastic case coincides with the result of the 
deterministic optimization (ignoring stochasticity) is the fact very well known in the control theory. j46| The optimal 
value of the functional in the deterministic case saturates to a constant at T — > oo, while in the stochastic case the 
average optimal cost grows with time linearly. Asymptotic convergence of the two seemingly different schemes to the 
same optimal control is thus an indication of the asymptotic self-consistency of the linear ansatz (|1.7I) . 

Differentiating Eq. p. 71) over 9 and then setting 8 to zero, one derives an alternative (to Eq. p. lip ) representation 
for the average rate of the cost function conditioned to K 

0% = / ^((^V-i+^V-VrQ) • (3.12) 

— OO 

Note that comparison of Eqs. (|3.7l3.11l3.12p also allows to derive expression for the derivative of the log-GF as a time 
integral, and then have it presented in an implicit algebraic form 

-f-oo 

- T- 1 d s log Z(0;K) = J ^-tr^V^+^V-^ + e" 

— OO 

= -tr ( VU] (3.13) 

2 V /M*n+mi=Q 



- J tr (Ve-^'tQe-^ dt, (3.14) 



where V = V(l + 6V '(/i*) -1 Q fJ. 



IV. OPTIMAL ASYMPTOTIC CONTROLS 



In this Section we formulate the RS-LQ, TO-LQ and CC-LQ asymptotic schemes in the general vector/matrix form 
as an optimization over K. (Note that the asymptotic LQ scheme was already stated as a minimum of Eq. ( 13.111) . 
or equivalcntly of Eq. (|3.12[) in the preceding Section.) Then we illustrate these formulations on the scalar example. 

/•fpjcj, which is asymptotically optimal for the RS-LQ control considered at 8 > 0, is found by maximizing Z(8] K). 
Using Eq. p.7p one derives 



mm 

K 



duj log 



dct (lo^- 1 + (BK - A)*V~~ 1 {BK - A) 



K*RK)) 



det (u; 2 ^- 1 + {BK - A)*V~ 1 {BK - A)) 



(4.1) 



Re(A(B_R"- J 4))>0 



where Ke(\(BK — A)) > denotes the stability condition ensuring that the real values of all the eigen- values of 
BK — A are positive. Note that constancy of the stationary RS-LQ optimal control was proven in [2a], therefore 
making our approach self-consistent. An alternative, but obviously equivalent, formulation of the RS-LQ optimal 
control consists in minimizing — T~ 1 dg log Z(8; K). Going along this path and utilizing Eq. p,13p one arrives at 



. 1 
mm — tr 

K,u 2 



(4.2) 
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generalizing the LQ formulation stated in the preceding Section as the minimization of Eq. (|3.11j) . Solving Eq. (|4.2|) 
is reduced to analysis of the respective generalization of the Riccati equations which can than be turned into a linear 
eigen- value problem described within the so-called Hamiltonian approach to the RS-LQ problem discussed in [28| . 

From Eq. (|1.5[) . and assuming time- independence of the control, one can state the general asymptotic TO-LQ 
optimum utilizing Eqs. (|3.2I3.7|) as an optimization of a double integral over frequency and 9. However, in practice 
one is interested to discuss the TO-LQ optimization only at sufficiently large values of the cost, jT. Using analysis of 
the preceding Section one derives the desired double asymptotic (valid at large T and large j) and simpler to state 
expression describing K^q 

nun max Re (6) Re (\( BK - A)) > > ( 4 .3) 

det Lfv-^n + eci} = o 

where max is over complex 9 and the optimal LD value of the PDF tail is exponential, 

logV TO (J)^-Rc(9 TO )J, ( 4 - 4 ) 

with ^xO s °l vm g Eq. I|3.10[) . Note that the det = condition in Eq. (|4.3|) is reminiscent of the ^-measure which is 
the key element of the robust control approach, see [35|, [3?J and references therein. 

In the same double asymptotic (large T and large j) regime the optimal CC-LQ control (|1.6j) is given by 



mintr (VII) Re(A(BA" - A)) > ( 4 .5) 

^*n + ikt 

)det(ii.*V- 1 ft+0Q)=a — JT 



max(Re(0)) detfu __ lu+e(3l=o > i^MiM 



Note that unlike Eqs. (|4.1l4.3p . Eq. (|4.5p does not have valid solutions for any value of the \og(l/e)/ J ratio. In fact, it 
is clear from Eq. (|4.4[) that to have a nonempty solution of Eq. (|4.5p one needs to require that Re(6>rpQ)jT < log(l/e). 
Once the optimum solution is found, one estimates the LD asymptotic of the cost function PDF by an expression 
similar to the one given by Eq. (|4.4j) . with TO subscript replaced by the CC one. 



A. Scalar case 



In the remainder of this Section we illustrate all of the aforementioned formulas on the scalar example. In this 
simple case integral on the rhs of Eq. ( I4.1[) is equal to 



2tt (y(BK - A) 2 + V6(Q + RK 2 ) - (BK - A)) , (4.6) 
resulting in the following optimal value 



A/B + ^A 2 /B 2 + Q/R + QV9/W 
Ke = 1 + VR9/B 2 ' (4J) 

The large deviation tail of the PDF of j at a given K can be extracted from Eq. (|3.9[) : 

(BK - A) 2 , , , % 

J»(J): ^)^ v(Q + RK^ +0 ^ (4 ' 8) 

Optimizing the PDF over K we find two different cases depending on the sign of A. At A > coefficient in front of 
the linear in j term on the rhs of Eq. (|4.8[) grows monotonically with K from the (A/B, +oo) interval. To find the 
optimal value of K in this case one has to take into the O(j) term thus deriving : 

.4>0: K T0 = ^4AjRV, logV T0 ^-^^. (4.9) 

In the other case of A = — \A\ < the linear coefficient in Eq. (|4.8[) reaches its maximum at K = BQ / (R\A\), thus 
resulting in 

A < : logP xo w (RA 2 + B 2 Q) . (4.10) 
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Finally, the CC-optimal formula (|4.5I) has no solution if B 2 j/(RV) < c in the A > case and if 
j (RA 2 + B 2 Q) /(RVQ) < c in the A < case. (Here we assume, as above, that e(0;T) = exp(-cT).) When 
e is chosen sufficiently small (i.e. c is sufficiently large), the feasibility domain in Eq. (|4.5|) is not empty and one 
distinguishes two regimes depending on how K s , defined by 




B 2 j 

compares with Kq, which is the bare LQ optimal value correspondent to Kg from Eq. (|4.7j) evaluated at 9 = 0. One 
derives 

K cc =me^(K e ,K ), (4.12) 

where of the two regimes one is achieved within the interior of the optimization domain (tail constraint is not 
restrictive) while the other one corresponds to the tail imposed by the boundary of the domain. It is worth noting 
that (|4.1ip is valid for both signs of A. 



V. EXAMPLE OF A STRING 



In this Section we discuss an explicitly solvable example of an infinite system where the set of zeros (of the 
determinant in the condition of Eq. (|3.10j) ) forms a quasi-continuous spectrum. Consider a string, defined as an 
over-damped system of multiple bids on a line connected to each other by elastic springs of strength D, stretched by 
the linear force of the strength A and subject to Langevien driving: 

d t Xj = Axj + D(xj + i + Xj_x — 2xj) + Biij + £j, (5-1) 
J= \j Q dt £(Gs? + fl«?)> (5-2) 

where j = 1, ■■■ ,N, Xj marks position of the j-th bid of the string, and the zero-mean white-Gaussian noise is 
distributed as in Eq. (|1.2[) with Vij = V6ij . Uj in Eq. (|5.ip stands for control. We are looking for a time-independent 
linear in x control, assuming that the control acts uniformly on all bids of the string, i.e. Uj — —Kxj. Let us also 
assume that the string is periodic with the period N. Then, solution of Eq. (|5.1[) allows expansion in the series over 
spatial harmonics 

N 

Xj = ^2exp(iq(j/N))x g , (5.3) 

with the wave vector, q, from the interval, — 7r < q < it, and resulting in the following separated equations for the 
individual harmonics 

d t x q = Ax q — 2D(1 — cosq)x q ~ BKx q + £ q . (5.4) 
Repeating the steps leading to (|3.8|) one arrives at 

log Z = | ( BK ~ A + 2D i l ~ cos _ V( BK - A + 2D i l - cosq)) 2 + V(Q + RK 2 )9j . (5.5) 

We choose to analyze only the most interesting regime, D 3> BK — A, when a nontrivial collective behavior emerges. 
Then, in the long wave-length, 1 — cosq — > q 2 /2, and continuous, J2 q ~^ (N/2ir) J dq, limits one derives 



2 i 



_ V(Q + RK 
S ~ {BK — A) 2 ' 

where one utilizes the standard IC 7 £ notations for the elliptic functions. 



(5.7) 
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Expression on the rhs of Eq. (|5.6[) shows a singularity at s = — 1, coinciding with the singularity (in the complex 
9 plane) observed in the scalar case at (9*. Substituting Eq. (|5.6|) into Eq. (|3.2j) and evaluating the integral over 9 in 
the saddle-point approximation one arrives at 

, . s/2 N(BK - A) 3 / 2 „ , . 

S * {J) W 3^1/2 J + (5-8) 



Juxtaposing the string expression Eq. (|5.8|l to the scalar one Eq. (|3.9|) one notes different behaviors with respect to 
BK — A. Optimizing Eq. (|5.8j) over if at a given large value J = jT, one obtains the same tail expression, second 
formula in (|4.9[) . however with another optimal control 

5/2 _ 2^nAD^ 
A str " RVT NB^ J ' [ ^ } 

replacing the first formula in Eq. (|4.9p . Note that in the string case the optimal K scales as J 2 / 5 which should be 
contrasted to the J 1 ' 2 scaling in the scalar case from Eq. (|4.9I) . 



VI. CONCLUSIONS AND PATH FORWARD 



This manuscript contributes the subject in control theory - designing control scheme with some guarantees not 
only on the average of the cost functions but also on fluctuations, specifically extreme fluctuations related to the tail 
of the cost function PDF. We consider linear, first order in time derivative, stochastic system of the Langevien type 
subject to minimization of a quadratic cost function and also with (chance) constraints imposed on the tail of the 
cost function PDF. In the stationary regime of large time, when control is sufficient to make the system stable, we 
reduce the stochastic dynamic problem of the "field theory" type to static optimization analysis with objectives and 
constraints stated in a matrix form. This type of reduction is unusual in the system lacking the fine-tuned Fluctuation 
Dissipation relation between relaxational and stochastic terms. On the other hand, the progress made is linked to 
linearity of the underlying stochastic systems which allowed, as in some problems of passive scalar turbulence [39T - l4l| 
and driven linear-elastic systems j38ll42|. to formally express solution for the system trajectory as an explicit function 
of the noise realization. Besides that, main technical ingredients, which allowed us to derive the results, consisted in 
making plausible assumption about the structure of the control (linear in the state variable and frozen in time) , and 
then performing asymptotic evaluations of the cost functions statistics conditioned to the value of the cost matrix. 
Techniques of path integral, spectral analysis and large deviation estimations were used. We tested results on the 
simple scalar case and illustrated utility of the method on an exemplary high-dimensional system (Id chain of particles 
connected in a string). 

We plan to continue exploring the interface between control theory and statistical physics addressing the following 
challenges. 

• Computational feasibility of the main formulas of the paper, stating RS-, TO- and CC- controls in 
Eqs. (|4.1l4.3l4.5p as static optimization problems, need to be analyzed for large systems and networks. Af- 
ter all main efforts in the applied control theory go into designing efficient algorithms for discovering optimal, 
or close to optimal, control, and we do plan to contribute this important task. Therefore, further analysis is 
required to answer the important practical question: if the static formulations of the newly introduced TO- 
QG and CC-QG controls allow computationally favorable exact or approximate expressions in terms of convex 
optimizations? 

• We also plan to study weakly non-linear stochastic systems through a singular perturbation stochastic diagram- 
matic technique of the Martin-Siggia-Rose type [43j . Besides, some of the methods we used in the manuscript, 
especially related to large deviation analysis, are not restricted to linear systems. Our preliminary tests show 
that effects of the non-linearity on the PDF tail are seriously enhanced in comparison with how the same 
nonlinear ity influences the average case control. 

• It will be interesting to study TO- and CC- versions of the path-integral nonlinear control problems discussed 
in (l3l - fl6l ]. These problems, in their standard min-cost formulations, allow reduction (under some Fluctuation- 
Dissipation-Thcorcm like relations between the form of control, covariancc matrix of the noise and the cost 
function) from the generally non-linear Hamilton- Jacobi-Bellman equations for the optimal cost function to a 
linear equation of a Schrodinger type. 

• The effects of partial observability and noise in the observations can be easily incorporated in both TO- and 
CC- schemes discussed in the paper. In fact this type of generalization is standard and widespread in the control 
theory, where for example the LQG (Linear-Quadratic-Gaussian) control generalizes the LQ control. 
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• In terms of relevance to an application, this work was motivated by recent interest and discussions related to de- 
veloping new optimization and control paradigms for power networks, so-called smart grids. In this application, 
strong fluctuations associated with loads and renewable generation, electro-mechanical control of generation, 
desire to make the energy production cheaper while also (and most importantly) maintaining probabilistic se- 
curity limitations of the chance-constrained type - all of the above make the theoretical model discussed in this 
paper an ideal framework to consider. In particular, we plan to extend the approaches of [3, El[ and modify 
and apply the theory developed in this manuscript to design a multi-objective Chance Constrained Optimum 
Power Flow including better control of generation, loads and storage resources in power grids. 

• We also anticipate that some of the models and results discussed in the paper are of interest for problems 
in statistical micro- and bio- fluidics, focusing on adjusting characteristics of individual molecules (polymers, 
membranes, etc) and also aimed at modifying properties of the medium (non-Newtonian flows) macroscopically. 
Time independent and linear nature of the control schemes discussed in the paper make them especially attractive 
for these applications. Natural constrains, e.g. associated with the force-field (optical or mechanical) as well 
as with some other physical limitations, could be incorporated into control as single- or multi-objective cost 
functions. 

We are thankful to D. Bicnstock, L. Gurvits, H.J. Kappen, K. Turitsyn and participants of the "Optimization and 
Control Theory for Smart Grids" project at LANL for motivating discussions and remarks. Research at LANL was 
carried out under the auspices of the National Nuclear Security Administration of the U.S. Department of Energy at 
Los Alamos National Laboratory under Contract No. DE C52-06NA25396. 
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